End-to-end evaluation in ATR-MATRIX: speech translation system between English and Japanese

ATR Interpreting Telecommunications Research Laboratories developed ATR-MATRIX speech translation system, which translates both ways between English and Japanese, enough to hold natural on-line real-time conversations. Using this system we started an end-to-end evaluation of a speech translation system through a dialog test with naive speakers who are not involved in system development and not familiar with a speech translation technology. This paper explains the speech translation design concept, evaluation system overview, evaluation procedure and some interesting results observed in the test. Finally after concluding we will mention our future plan.